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APPENDIX  A 

A  DESCRIPTION  OF  INTERNAL  RELATIONSHIPS 

A.  1  INTRODUCTION 

All  internal  relationships  included  in  this  prototype 
model  have  their  foundation  in  data  analysis.  This  does  not 
imply  that  all  relationships  are  statistically  valid.  The 
amount  and  the  quality  of  the  data  supporting  the  relation¬ 
ships  vary  greatly.  In  cases,  some  degree  of  statistical 
significance  does  exist.  More  frequently,  data  was  sufficient 
only  to  identify  trends  and  general  relationships.  From  these 
identifiable  trends,  mathematical  formulations  were  derived 
which  captured  these  relationships.  For  the  most  part,  the 
methodology  adopted  is  a  heuristic,  empirical  one,  which  lends 
itself  to  validation  and  modification  through  rigorous  data 
analysis.  That  is  to  say,  additional  data  can  be  used  to 
accept,  reject, or  modify  many  of  the  postulated  relationships. 


A. 2  DEVELOPMENT  PHASES 

A. 2.1  Technical  Risk  Reduction 

The  input  estimates  of  technical  risk  and  their 
levels  of  confidence  are  transformed  into  a  vector  repre¬ 
senting  the  probability  distribution  of  risk  for  each  cate¬ 
gory.  The  probability  distribution  is  based  upon  the  binomial 
distribution.  The  estimated  level  of  risk  determines  the 
mean,  and  the  level  of  confidence  determines  the  dispersion 


A- 1 


THE  ANALYTIC  SCIENCES  CORPORATION 


about  the  aean.  The  process  used  for  accomplishing  this  trans¬ 
formation  is  arbitrary,  but  accomplishes  the  desired  result  of 
incorporating  uncertainty  into  the  risk  assessment  process. 

The  risk  reduction  matrices  generated  from  the  ques¬ 
tionnaire  (see  Appendix  E)  constitute  linear  transformations 
or  mappings.  These  risk  matrices  differ  by  risk  category  and 
strategy /phase  alternative.  Ordinary  matrix  multiplication 
results  in  a  modified  description  of  technical  risk  by  risk 
category  and  by  strategy  alternative. 

The  set  of  risk  vectors  existing  at  the  completion  of 
Phase  2  (which  are  a  function  of  risk  category  and  development 
strategy)  represents  the  probability  distribution  of  technical 
risk  remaining  at  the  completion  of  FSD  given  a  particular 
multi-phase  development  strategy.  These  are  used  later  in  the 
calculation  of  probabilities  of  success. 

A. 2. 2  Time  and  Cost 

Since  the  results  of  the  data  analysis  indicated  a 
high  correlation  between  the  cost  of  pursuing  a  phase  alter¬ 
native  and  the  level  of  risk  at  the  beginning  of  that  phase, 
this  phenomenon  was  included  in  the  model.  The  cost  of  pur¬ 
suing  each  phase  alternative  is  described  in  terms  of  a  normal 
(Gaussian)  probability  distribution  with  mean  and  standard 
deviation  determined  from  the  data  analysis  after  removing 
the  influence  of  technical  risk.  The  mean  and  standard  devia¬ 
tion  are  then  weighted  by  the  three  probability  distributions 
of  risk  calculated  for  the  beginning  of  that  phase  alter¬ 
native.  Each  risk  category  received  equal  weighting.  Since 
the  phase  alternatives  reduce  risk  differently,  the  cost  asso¬ 
ciated  with  pursuing  each  alternative  varies  accordingly  (in 
addition  to  the  differences  in  the  cost  which  are  a  function 
of  phase  alternative  only). 
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The  time  required  to  complete  each  phase  alternative 
is  represented  as  a  normal  probability  distribution  with  mean 
and  standard  deviation  derived  from  the  data  analysis.  Little 
correlation  between  time  and  risk  was  evident  in  the  data. 

The  time  and  cost  associated  with  the  complete  devel¬ 
opment  effort  (all  phases)  are  derived  from  the  probability 
distribution  of  the  constituent  phases  according  to  standard 
rules  of  probability  theory.  The  probability  distributions  of 
time  are  used  in  calculating  the  probability  of  success,  and 
the  probability  distributions  of  cost  are  used  in  the  relative 
cost  comparisons. 

A. 2. 3  The  Effect  of  Concurrency 


The  degree  of  concurrency  impacts  not  only  the  risk 
reduction  matrices  derived  from  the  questionnaire  described  in 
Appendix  E,  but  two  additional  areas  as  well. 

The  first  area  affected  is  time  to  Initial  Opera¬ 
tional  Capability  (discussed  in  Appendix  D).  The  higher  the 
degree  of  concurrency,  the  sooner  IOC  is  realized.  A  normal 
probability  distribution  is  assumed  with  the  mean  and  standard 
deviation  determined  from  data  analysis. 

The  second  area  is  the  effect  on  production  options. 
The  model  assumes  that  it  is  not  possible  to  develop  a  second 
production  source  during  the  period  of  concurrency  when  FSD 
and  early  production  activities  overlap.  The  higher  the 
degree  of  concurrency,  the  longer  the  delay  before  a  second 
production  source  becomes  feasible.  This  degree  of  overlap 
between  FSD  and  production  was  derived  from  programs  employing 
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concurrency  and  is  represented  in  the  model  by  a  normal  proba¬ 
bility  distribution  with  mean  and  variance  determined  from  the 
data  analysis. 


A. 3  PRODUCTION  PHASE 

The  majority  of  the  relationships  incorporated  into 
the  production  phase  stem  from  the  Production  Cost  Analysis 
Methodology  (PCAM)  previously  developed  by  TASC.  Data  col¬ 
lected  during  this  effort,  combined  with  data  previously  ana¬ 
lyzed,  were  used  to  generate  two  generic  sets  of  PCAM  para¬ 
meters,  one  suitable  for  tactical  missile  systems  and  one  suit¬ 
able  for  electronic  subsystems.  A  detailed  discussion  of  PCAM 
is  provided  in  Appendix  B. 

There  is  one  additional  aspect  relating  to  production 
cost  that  became  evident  during  the  data  analysis  effort. 

That  aspect  concerns  the  effect  of  competitive  FSD  on  subse¬ 
quent  production  cost.  Although  the  amount  of  supporting  data 
is  currently  small,  there  is  clear  evidence  that  competition 
during  FSD  suppresses  (compared  to  single  source  FSD)  subse¬ 
quent  production  costs,  at  least  for  the  first  few  years. 

This  concept  is  incorporated  in  the  model,  but  additional 
research  is  warranted. 


A. 4  PROBABILITY  OF  SUCCESS  CALCULATION 

The  probability  that  a  given  acquisition  strategy 
meets  the  IOC  requirement  at  a  level  of  design  stability  is 
the  combination  of  the  set  of  risk  vectors  existing  at  the 
completion  of  FS  (see  s*"  tion  A. 2)  with  the  probability  that 
the  multi-phase  ae»r-  top.  ent  strategy  can  be  completed.  The 
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set  of  risk  vectors  are  used  as  conditional  probabilities 
representing  the  probability  that  technical  risk  is  reduced  to 
the  specified  level,  given  the  development  phase  is  completed. 

The  calculation  further  assumes  that  the  risk  reduc¬ 
tion  process  across  the  three  categories  of  technical  risk  are 
independent.  It  is  recognized  that  a  certain  amount  of  syner¬ 
gism  exists;  however,  arriving  at  an  appropriate  methodology 
for  including  these  perceived  interdependenices  is  a  non¬ 
trivial  undertaking.  TASC  recommends  that  this  research  be 
performed  in  order  to  add  increased  validity  and  realism  to 
the  model. 

Under  these  assumptions,  the  probability  of  success 
is  calculated  by  multiplying  all  constituent  probabilities. 

The  principal  inadequacy  resulting  from  this  approach  is  that 
the  probabilities  of  success  appear  lower  than  expected.  They 
appear  adequate,  however,  as  relative  indicators  which  is  their 
purpose . 


A. 5  DOMINANCE  ANALYSIS 

At  this  point  in  the  evaluation  process,  ten  prin¬ 
cipal  attributes  describe  each  strategy:  six  probabilities  of 
success  (3  levels  and  two  IOC  times),  relative  development 
cost,  and  relative  total  program  cost  for  three  inventory  esti¬ 
mates.  In  this  form,  they  are  all  on  a  scale  from  zero  to 
one.  For  the  probabilities  of  success,  greater  is  superior; 
for  the  relative  cost  estimates,  smaller  is  superior. 

The  first  step  in  the  dominance  analysis  is  a  pair¬ 
wise  dominance  comparison  incorporating  a  "closeness  criteria". 
For  a  closeness  parameter  of  a  (.03,  .05,  and  .07  are  all  used), 
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strategy  A  is  said  to  dominate  strategy  B  if  and  only  if  (1) 
no  attribute  of  B  is  superior  to  the  corresponding  attribute 
of  A  by  an  amount  as  great  as  a,  and  (2)  at  least  one  attri¬ 
bute  of  A  is  superior  to  the  corresponding  attribute  of  B  by 
an  amount  at  least  as  great  as  a.  From  this  comparison,  an  n 
x  n  matrix  (where  n  equals  the  number  of  strategies  being  com¬ 
pared)  is  produced  for  each  a.  If  (6ij)  denotes  the  matrix, 
then  6ij  is  1  if  and  only  if  strategy  i  dominates  strategy  j; 
otherwise,  6ij  =  0  (note  that  6ii  =  0  for  all  i). 

Note  that  with  this  "soft"  aspect  of  dominance,  it  is 
possible  for  strategy  A  to  dominate  strategy  B,  for  strategy  B 
to  dominate  strategy  C,  and  for  strategy  C  to  dominate  stra¬ 
tegy  A.  Because  of  this,  a  strategy  must  pass  two  criteria  to 
be  considered  to  be  dominated  by  the  set  of  strategies  and 
eliminated  from  further  consideration:  (1)  it  must  be  domi¬ 
nated  by  at  least  one  strategy  in  the  set,  and  (2)  it  must  not 
dominate  any  other  strategy  in  the  set.  Any  strategy  which 
meets  both  criteria  can  be  considered  as  "inferior"  or  a 
"second-best"  option  and  eliminated  from  further  considera¬ 
tion. 

If  one  or  more  strategies  are  eliminated  in  this 
fashion,  the  subset  of  strategies  remaining  constitute  a  new 
set  of  strategies  for  the  "set  comparison",  and  the  process 
can  be  repeated.  The  entire  process  is  iterated  until  no 
further  strategies  pass  the  two  criteria.  The  resulting  set 
of  strategies  then  contains  no  "second-best"  options. 

This  process  is  performed  for  three  "closeness  cri¬ 
teria".  The  choice  of  the  resultant  set  displayed  is  made 
arbitrarily  by  a  heuristic  algorithm.  If  a  more  stringent 
closeness  criteria  than  the  one  selected  for  display  would 
result  in  additional  strategies  being  eliminated,  a  message  to 
this  effect  is  printed. 
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As  mentioned  in  the  report,  the  process  should  be 
considered  experimental.  Further  research  and  analysis  into 
the  implications  of  this  procedure,  as  well  as  possibly  alter¬ 
native  methodologies,  is  justified. 
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APPENDIX  B 

THE  TASC  PRODUCTION  COST 
ANALYSIS  METHODOLOGY 

The  TASC  Production  Cost  Analysis  Methodology  (PCAM) 
integrates  data  analysis  with  management  analysis  and  computer 
models.  It  is  used  to  assist  government  program  personnel  in 
assessing  the  relative  costs  and  benefits  of  alternative 
acquisition  approaches.  A  conceptual  description  of  PCAM  is 
shown  in  Figure  B-l. 


Figure  B-l  PCAM  Description 
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B . 1  DATA  ANALYSIS 

TASC's  data  analysis  considers  both  program  specific 
data  and  historical  data  on  similar  programs.  Consideration 
of  program  specific  production  parameters  enables  TASC  to 
tailor  the  methodology  to  the  unique  needs  of  a  particular 
program .  Data  on  similar  programs  is  analyzed  to  establish  a 
historical  basis  for  the  cost  estimation.  TASC  uses  several 
computer  programs  to  assist  in  analyzing  the  raw  data.  These 
incorporate  techniques  such  as  linear  regression  and  least 
squares  curve  fitting.  The  results  of  the  data  analysis  are 
used  to  identify  cost  improvement  curve  relationships,  produc 
tion  rate  characteristics,  and  the  impact  of  competition. 


B . 2  COST  IMPROVEMENT  CURVE 

A  cost  improvement  curve  reflects  the  relationship 
between  the  unit  cost  (or  unit  price)  of  an  item  and  the 
quantity  of  the  item  produced.  An  "80  percent"  curve  is  one 
in  which  a  doubling  of  output  drives  unit  cost  down  to  80 
percent  of  its  initial  value.  That  is,  the  cost  of  the  2Nth 
unit  is  20  percent  less  than  the  cost  of  the  Nth  unit.  Graph 
ically  displayed  in  Figure  B-2  is  a  cost  improvement  curve  in 
standard  form. 
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Figure  B-2  Cost  Improvement  Curve  in  Standard  Form 

Frequently,  cost  improvement  curves  are  depicted  in 
logarithmic  form  (the  logarithm  of  unit  cost  as  a  function  of 
the  logarithm  of  cumulative  quantity)  producing  the  linear  rela 
tionship  display  in  Figure  B-3. 


Figure  B-3  Cost  Improvement  Curve  in  Log-Linear  Form 
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B. 3  COMPETITION 

TASC  has  identified  two  effects  that  the  introduction 
of  competition  has  on  the  unit  cost  improvement  curve.  One  of 
these  is  a  downward  shift  in  the  curve  resulting  in  an  immed¬ 
iate  decrease  in  unit  costs.  The  other  is  a  rotation  of  the 
curve,  or  an  increased  rate  of  cost  improvement  for  all  future 
units  produced.  Figure  B-4  illustrates  the  total  effect  of 
competition  on  the  log- linear  form  of  the  cost  improvement 
curve . 


Figure  B-4  Benefits  of  Competition 


TASC's  prior  analysis  of  several  tactical  missile 
programs  clearly  demonstrated  the  shift  and  rotation  of  the 
cost  improvement  curve  due  to  competitive  forces.  Recent 
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analysis  has  demonstrated  that  the  magnitudes  of  the  shift  and 
the  rotation  vary.  These  variations  are  highly  correlated  both 
with  the  number  of  units  produced  on  a  non-competitive  basis 
and  with  the  slope  of  the  non-competitive  cost  improvement 
curve.  From  this,  TASC  hypothesized  a  theoretical  framework 
which  correlates  with  the  observed  results  as  shown  in  Figure 
B-5.  The  hypothesis  incorporated  an  "optimal”  or  "best"  cost 
improvement  curve  that  one  might  observe  if  the  manufacturer 
were  under  continuous  competitive  pressure  from  the  outset. 

Detailed  analysis  was  conducted  on  five  missile  cases 
to  test  the  hypothesized  framework.  The  results  of  the  analy¬ 
sis  revealed  that,  on  the  average,  the  "optimal"  curve  could 
be  achieved  with  no  shift  in  the  first  unit  cost  and  a  four 
percent  rotation  of  the  cost  improvement  curve  at  the  origin, 
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Figure  B-5  TASC  Framework:  The  Impact  of  Competition 
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assuming  the  manufacturer  was  under  continuous  competitive 
pressure.  Applying  linear  regression  with  the  non-competitive 
slope  as  the  independent  variable  and  the  optimal  competitive 
slope  as  the  dependent  variable  yields  a  significant  correla¬ 
tion  coefficient  of  .995  for  the  five  missile  cases  examined. 

The  data  analysis  demonstrated  that  the  flatter  the 
non-competitive  curve  and  the  larger  the  quantity  produced 
prior  to  competition,  the  greater  the  potential  for  unit  cost 
reductions  from  the  original  producer  once  competition  is 
introduced.  The  observed  shift  and  rotation  of  the  original 
producer's  cost  improvement  curve  resulting  from  competitive 
pressure  can  be  characterized  as  "making-up"  for  cost  improve¬ 
ments  which  were  possible  but  were  unrealized  due  to  the 
absence  of  competitive  pressure.  The  earlier  competitive  pres¬ 
sure  is  applied  during  the  production  phase,  the  earlier  the 
producer  attains  his  optimal  curve,  and  the  greater  are  the 
total  savings  due  to  competition. 

The  obvious  source  of  competitive  pressure  is  price 
competition  by  the  second  source.  TASC's  continuing  analysis 
has  demonstrated  that  the  relationships  between  the  first  and 
second  producers'  cost  behavior  are  as  displayed  in  Figure 
B-6. 


The  first  source  had  a  first  unit  cost  represented  by 
point  A  and  followed  the  cost  improvement  curve  so  designated. 
After  units  were  produced  by  the  first  source,  a  second 
source  began  production  with  a  first  unit  cost  of  B  and  fol¬ 
lowed  the  cost  improvement  curve  represented  by  the  dashed 
line.  In  each  of  the  five  missile  cases,  B  was  less  than  A 
and  greater  than  C,  the  unit  price  achieved  by  the  first 
source  after  items  has  been  produced.  Also  in  each  of  the 
five  cases,  the  slope  of  the  second  source's  cost  improvement 
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Figure  B-6  Cost  Improvement  Curves  of 
First  and  Second  Source 

curve  was  steeper  than  the  slope  of  the  first  source's  cost 
improvement  curve.  On  the  average,  the  second  source  cost 
improvement  curves  were  5  percent  steeper  than  the  first 
source  curves,  and  the  first  unit  cost  of  the  second  source 
(B)  was  25  percent  less  than  the  first  unit  cost  of  the  first 
source  (A). 

For  the  cases  analyzed,  the  implication  is  that  the  second 
source  begins  production  at,  or  very  near,  his  "optimum"  cost 
improvement  curve.  This  enables  the  second  source  to  rapidly 
approach  cost  parity  with  the  first  source  during  the  production 
of  his  learning  quantities.  Following  the  learning  buys,  com¬ 
petition  is  introduced  and  the  first  source  shifts  and  rotates 
his  cost  improvement  curve  by  amounts  sufficient  to  bring  his 
price  slightly  below  that  of  the  second  source. 

B-7 


THE  ANALYTIC  SCIENCES  CORPORATION 


B.4  PRODUCTION  RATE  VARIATIONS 

Prior  research  into  the  effect  of  production  rate  on 
unit  cost  has  shown  that  unit  cost  frequently  decreases  with 
increasing  production  rate  in  a  form  virtually  identical  to 
that  of  cost  improvement  curves.  However,  traditional  econo¬ 
mic  theory  says  that  there  are  both  economies  and  diseconomies 
of  scale,  and  the  curve  should  be  U-shaped  if  one  assumes  pro¬ 
duction  capacity  to  be  fixed.  The  TASC  formulation  embodies 
both  of  these  considerations  as  depicted  in  Figure  B-7. 

The  formulation  assumes  the  existence  of  an  optimum 
(most  cost  efficient)  production  rate,  denoted  Rq  in  Figure 
B-7.  Typically,  a  manufacturer  will  arrive  at  this  rate  in  an 
attempt  to  maximize  profits  by  considering  his  facility  limita¬ 
tions,  capital  investment  requirements,  anticipated  quantities 
to  be  procured  by  the  government,  the  rates  of  procurement,  and 
other  requirements  specified  by  the  government. 


UNIT 

COST 


Figure  B-7  Effect  of  Production  Rate  on  Unit  Cost 
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It  was  further  assumed  that  the  curve  should  be  symmetric  about 
Rq.  Thus,  production  costs  are  minimized  when  the  production 
rate  is  equal  to  Rq,  and  increase  as  one  deviates  from  Rq  in 
either  direction.  This  formulation  imposes  the  restriction 
that  the  maximum  allowable  production  rate  is  (2  x  Rq)-1.  For 
a  manufacturer  to  produce  at  a  higher  rate,  the  production 
capacity  would  have  to  be  expanded,  thus  producing  a  new  Rq. 

It  should  be  noted  that  the  symmetric  shape  of  the 
production  rate  curve  is  an  assumed  shape.  Other  authors  on 
the  subject  may  contend  that  a  curve  of  one  shape  (possibly 
the  one  chosen)  should  be  used  as  one  increases  production 
rate  up  to  the  optimal  value,  and  that  a  curve  of  a  different 
shape  be  used  as  one  increases  production  rate  above  the 
optimal  value.  TASC's  data  analysis  has  indicated  that  the 
specific  shape  of  the  production  curve  is  unique  for  each  pro¬ 
gram,  determined  by  particular  program  and  contractor  charac¬ 
teristics.  The  flexibility  of  TASC's  methodology  enables  it 
to  incorporate  these  unique  aspects.  Detailed  analysis  of  the 
five  missile  cases  has  indicated  that  the  general  symmetric 
production  rate  curve  can  be  employed  to  predict  production 
costs  within  one  percent  of  actual  costs. 


B . 5  COST  CALCULATIONS 

Combining  the  effects  of  both  competition  and  produc¬ 
tion  rate  into  a  unified  model  results  in  the  TASC  Cost  Improve 
ment  and  Production  Curve  (CIPC)  model.  Figure  B-8  displays 
the  resulting  relationships  graphically. 
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Cost  estimates  for  alternative  acquisition  approaches 
for  individual  programs  are  generated  using  the  CIPC  model.  It 
simultaneously  considers  cost  improvement  curve  effects,  pro¬ 
duction  rate  variations  and  the  impact  of  competition  on  the  re 
curring  production  costs  of  weapon  systems  and  subsystems.  The 
basic  formulation  considers  unit  production  cost  as  a  function 
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of  both  total  quantity  produced  and  the  production  rate. 
Mathematically,  this  is  expressed  as  follows: 

Z  =  AXBYC 

where 

Z  =  unit  cost  of  the  item  produced 

A  =  constant  (sometimes  referred  to  as  T, 
or  "first  unit  cost") 

X  =  cumulative  quantity  produced 

B  =  coefficient  which  describes  the  slope 
of  the  quantity/cost  curve 

Y  =  (proxy)  production  rate  in  effect 

C  =  coefficient  which  describes  the  slope 
of  the  rate/cost  curve. 

B . 6  MANAGEMENT  ANALYSIS 

Sensitivity  and  management  analyses  can  be  performed 
to  assess  alternative  acquisition  strategies  in  relation  to 
unique  program  characteristics.  Factors  such  as  timing  of 
competition,  cost,  schedule  and  technical  risk,  product  im¬ 
provement,  lower  tier  contractor  development,  and  contracting 
alternatives  are  frequently  considered.  This  ensures  that  the 
analysis  considers  all  the  factors  that  are  relevant  to  a  par¬ 
ticular  program's  acquisition  approach  evaluation.  In  turn, 
the  results  of  these  analyses  enable  program  personnel  to  make 
informed  decisions  when  selecting  an  acquisition  approach. 


I 

i 


B-l  1 


V- 


THE  ANALYTIC  SCIENCES  CORPORATION 


B. 7  SUMMARY 

The  TASC  PCAM  is  a  results-oriented  acquisition  re¬ 
search  and  analysis  methodology  that  incorporates  detailed 
data  analysis  with  management  analysis,  computer  models  and 
cost  estimating.  PCAM  was  developed  to  assist  government 
program  offices  in  assessing  the  relative  costs  and  benefits 
of  alternative  acquisition  approaches.  PCAM  is  a  dynamic 
methodology  that  enables  TASC  to  address  the  unique  require¬ 
ments  of  a  particular  program  as  well  as  incorporate  the 
changing  characteristics  of  the  DoD  acquisition  process. 

PCAM  is  a  proven  analytic  tool.  It  has  been  success¬ 
fully  used  to  support  programs  such  as  the  Cruise  Missiles, 
AMRAAM,  and  the  Global  Positioning  System  User  Equipment. 
PCAM's  dynamic,  practical,  and  results  oriented  nature  makes 
it  applicable  to  any  major  system  or  subsystem  acquisition 
program. 
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APPENDIX  C 
DATA  COLLECTION 


The  development  of  the  prototype  model  required  a 
large  and  varied  amount  of  data  covering  the  four  acquisition 
phases.  Two  major  categories  of  data,  program  history  and 
judgement  data,  were  required.  The  first  category  centered  on 
specific  program  parameters: 

•  Strategy  used  in  acquisition  phase 

•  Length  of  phase 

•  Cost  of  phase 

•  Risk  levels  prior  to  Phases  1  and  2 

•  Interim  between  start  of  Phase  2  and 
IOC 

•  Overlap  between  production  and 
development 

•  Second  source  start-up  costs 

•  Learning  curve  parameters 

•  First  unit  cost 

•  Production  rate  parameters. 

The  second  category  dealt  with  measuring  the  ability  of  a 
selected  strategy  to  reduce  technical  risks  to  a  manageable 
level.  Each  category  had  a  different  data  collection  process, 
level  of  difficulty,  and  degree  of  success. 
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C.l  PROGRAM  HISTORY  COLLECTION 

The  data  collection  effort  of  selected  program  his¬ 
tories  was  limited  by  the  following  factors: 

•  Scope  of  the  model 

•  Collection  process 

•  Quality  and  quantity  of  data. 

Each  factor  individually  and  collectively  affected  the  sources 
and  types  of  data  collected. 

The  scope  of  the  prototype  model  was  limited  to  tac¬ 
tical  missile  systems  and  military  electronics  subsystems. 

These  categories  were  selected  because  of  their  broad  base 
within  the  military  community.  For  example,  both  categories 
of  systems  are  developed  and  produced  in  all  three  services: 
thus,  the  model  would  be  relevant  to  a  large  number  of  sectors 
in  each  service.  It  was  also  felt  that  the  extensive  use  of 
electronic  subsystems  within  tactical  missiles  provided  an  inter¬ 
dependent  relationship  that  would  be  useful  in  data  collection 
and  validation  of  results.  In  addition,  TASC's  previous  re¬ 
search  efforts  had  resulted  in  an  accumulation  of  missile  and 
electronics  production  cost  data.  It  was  felt  that  this  exist¬ 
ing  data  would  reduce  the  required  data  collection  effort. 

The  intent  of  the  development  effort  further  limited 
the  scope  of  the  data  collection  effort.  The  goal  of  the  phase 
was  to  produce  a  model  capable  of  successfully  demonstrating 
the  ability  to  model  the  acquisition  environment  in  a  useful 
way.  Thus,  efforts  were  limited  to  levels  necessary  to  demon¬ 
strate  the  model's  feasibility.  Additional  data  would  greatly 
enhance  the  rigor  of  estimating  the  model's  parameters. 
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The  data  collection  process  followed  a  long  and  some¬ 
what  tedious  path  depicted  in  Figure  C-l.  As  shown,  data 
leads  were  sought,  and  after  a  period  of  time,  usually 
located.  Typically,  a  number  of  calls  were  required  to  locate 
a  knowledgeable  and  responsible  individual  with  control  over 
access  to  the  data.  After  the  request  for  access  was  verbally 
agreed  to,  a  formal  request  from  DSMC  was  required.  After 
this  request  had  worked  its  way  through  the  chain  of  command 
to  the  person  coordinating  our  request,  an  appointment  could 
be  made  and  clearances  sent.  This  process  spanned  approxi¬ 
mately  two  to  six  months  and  required  conversations  with  a 
large  number  of  people  to  obtain  necessary  permission.  The 
length  of  this  process  dominated  the  process  and  highlighted 
the  quantity  and  quality  of  data  resulting  from  each  iteration. 


Figure  C-l.  Data  Collection  Process 

The  original  data  collection  effort  was  directed 
towards  locating  and  accessing  individual  missile  and  elec¬ 
tronic  subsystem  program  offices.  The  objective  was  to  obtain 
a  basically  complete  and  useful  history  of  the  specific  pro¬ 
gram  through  interviews  and  examination  of  program  records  and 
documents.  The  quality  and  quantity  of  data  obtained  in  this 
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way  varied  greatly.  In  all  cases,  the  program  offices  kept 
records  of  the  acquisition  phase  currently  in  progress.  Some 
maintained  records  of  prior  phases  ranging  from  a  one  para¬ 
graph  history  to  major  summaries.  Thus,  in  most  cases,  only 
one  phase  was  sufficiently  covered,  with  additional  effort 
required  to  locate  and  obtain  information  about  earlier 
activities . 

The  length  of  time  required  to  obtain  the  individual 
program  office  data  limited  the  amount  of  data  that  could  be 
collected.  In  an  effort  to  increase  the  volume  of  data,  col¬ 
lection  efforts  were  redirected  towards  the  Historical  Offices 
of  the  various  missile  and  electronics  commands  (while  still 
following  up  on  prior  program  office  contracts).  One  of  the 
primary  functions  of  these  offices  is  to  collect,  analyze,  and 
synthesize  weapons  development  and  production  histories  from 
concept  exploration  to  end  of  production.  Each  office  con¬ 
tained  this  information  on  many  programs,  thus  reducing  the 
length  of  data  collection  per  program.  In  addition,  these 
data  were  the  most  complete  source  of  individual  program 
information.  Their  only  drawbacks  were  the  ages  of  the  pro¬ 
grams  summarized. 

In  addition  to  program  office  and  Historical  Office 
data,  two  other  less  significant  data  sources  were  tapped. 

The  Selected  Acquisition  Reports  (SAR)  collected  on  major  pro¬ 
grams  could  be  used  alone  or  as  supplements  to  other  infor¬ 
mation.  More  research  in  this  area  would  be  beneficial.  The 
second  source  came  from  prior  studies  and  analyses.  These 
also  could  be  used  alone:  for  example,  TASC's  production  rate 
work,  or  as  a  supplement,  SAI ' s  collection  of  R&D  cost/time 
summaries . 


C-4 


THE  ANALYTIC  SCIENCES  CORPORATION 


In  sum,  the  data  collected  from  these  four  sources 
was  adequate  to  develop  the  specific  program  parameters  neces¬ 
sary  to  the  prototype  model.  Information  on  alternatives  was 
collected  from  thirty-seven  programs  for  one  to  four  phases 
(see  Table  C-l).  In  addition,  other  program  histories  were 
collected  that  were  insufficient  for  full-phase  analysis  but 
lent  insight  into  the  analysis  and  estimation  of  the  model 
parameters . 


C . 2  RISK- REDUCTION  MEASUREMENT 

During  the  feasibility  study,  the  method  of  quanti¬ 
fying  risk  reduction  was  developed.  Given  this  basis,  a  ques¬ 
tionnaire  was  chosen  as  the  best  way  of  obtaining  the  subjec¬ 
tive  assessments  from  knowledgeable  individuals.  The  initial 
analytic  approach  used  The  Analytic  Hierarchy  Process  (AHP) 
developed  by  Thomas  L.  Saaty  of  the  University  of  Pittsburgh. 
This  method  proved  to  be  much  too  difficult  for  most  indivi¬ 
duals  not  familiar  with  AHP  or  the  definitions  embodied  in  the 
questionnaire.  After  numerous  iterations  to  reduce  the  com¬ 
plexity  of  the  questionnaire,  the  method  was  abandoned. 


The  method  finally  used  is  shown  in  Appendix  D.  While 
this  new  questionnaire  was  simpler  than  the  first  it  was  still 
extremely  difficult  for  the  respondents  unless  verbally  admini¬ 
stered  to  them.  The  type  of  information  needed  limits  the 
amount  of  simplification  available.  The  prototype  model  would 
benefit  from  the  refinement  resulting  from  additional  question- 
naire  administration. 
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APPENDIX  D 
DATA  ANALYSIS 


The  purpose  of  the  data  analysis  was  to  estimate  the 
model  parameters  required  to  describe  each  strategy  alterna¬ 
tive  for  each  of  the  four  acquisition  phases.  Primary  empha¬ 
sis  was  placed  on  calculating  these  parameters  from  collected 
program  histories.  For  those  cases  with  little  or  no  data, 
these  strategy  parameters  were  estimated  from  the  parameters 
derived  from  the  program  data.  The  basic  methods  used  to 
derive  the  necessary  parameters  were  different  for  the  devel¬ 
opment  phases  than  for  the  production  phase.  The  methodology 
and  results  follow. 


D.l  DEVELOPMENT  PHASE  ANALYSIS 

The  first  step  in  the  analysis  was  to  determine  the 
strategy  alternatives  used  throughout  the  program  and  their 
approximate  start  and  end  dates.  The  selection  of  the  types 
of  strategy  alternatives  was  based  on  the  program  strategy  al¬ 
ternatives  described  in  Chapter  Two  of  the  feasibility  study 
(TR-1375).  Descriptions  of  program  actions  and  plans  recorded 
in  program  files,  as  well  as  outside  discussions  found  in 
various  journals  and  other  publications,  were  compared  to 
these  definitions.  The  best  descriptive  fit  was  then  chosen. 
Estimation  of  phase  start  and  end  dates  was  made  in  a  number 
of  ways.  In  later  weapons  programs,  DSARC  or  individual  ser¬ 
vice  review  dates  were  available,  and  used  for  the  start  and 
end  dates.  In  earlier  programs  not  covered  under  this  review 
process,  start  and  end  dates  were  based  on  contractual  infor¬ 
mation,  program  memoranda,  proposed  schedules  and  plans,  and/or 
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activity  descriptions.  Another  tine  parameter  needed  was  the 
date  of  initial  operational  capability  (IOC).  In  most  cases, 
an  IOC  date  was  included  in  program  documentation.  Where  no 
record  of  an  actual  IOC  date  was  available,  IOC  was  estimated 
to  be  twelve  months  after  first  production  delivery.  The 
length  of  each  phase  alternative  was  calculated  in  months  from 
the  start  and  end  dates.  Time  to  IOC  was  described  in  months 
from  start  of  Phase  2. 

After  delineating  each  program  by  length  and  type  of 
activity,  the  cost  associated  with  each  strategy  alternative 
was  determined.  In  some  instances,  cost  records  were  avail¬ 
able  by  fiscal  year  and  type  of  money.  These  monies  were  con¬ 
verted  to  FY80  dollars  and  aggregated  over  the  correct  phase 
of  the  program.  In  other  programs,  where  this  type  of  infor¬ 
mation  was  not  available  or  incomplete,  contract  dollars 
and/or  SAR  records  were  used.  These  monies  were  aggregated 
into  the  proper  program  phases  by  dates  and  listed  require¬ 
ments  . 


An  additional  piece  of  required  data  was  the  risk 
level  prior  to  Phases  1  and  2.  Using  the  risk  categories  and 
definitions  developed  for  the  questionnaire,  each  phase  alter¬ 
native  was  assigned  a  risk  level  based  upon  technical  descrip¬ 
tions  and  reports  of  program  personnel  and  knowledgeable  indi¬ 
viduals  outside  the  program. 

The  results  of  the  above  analyses  were  collected  into 
sets  of  cost,  time,  and  risk  for  each  strategy  alternative 
represented  by  data.  A  mean  and  standard  deviation  of  time 
and  cost  per  unit  of  risk  were  calculated.  Those  strategy 
alternatives  not  represented  in  the  data  sample  were  then 
estimated  from  these  points.  A  good  example  of  a  well 
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represented  strategy  alternative  was  the  Phase  1  -  Prototype/Single 
Source  alternative.  Table  D-l  shows  the  data  and  results  for 
this  alternative  for  tactical  missiles. 


TABLE  D-l 


PHASE  1 

-  PROTOTYPE/SINGLE 

SOURCE 

Case 

Months 

Cost  (MIL  80$) 

Risk 

Cost/Risk 

1 

60 

25.96 

4 

6.49 

2 

16 

12.32 

2 

6.16 

3 

60 

106.62 

6 

17.77 

4 

24 

64.70 

6 

10.78 

5 

49 

60.11 

4 

15.03 

6 

17 

11.96 

2 

5.98 

7 

62 

74.75 

5 

14.95 

Mean 

41.14 

50.92 

11.02 

S.D. 

21.14 

32.90 

4.90 

D. 2  PRODUCTION  PHASE  ANALYSIS 

Three  types  of  parameters  were  required  for  each 
alternative  in  the  missile  and  electronic  subsystem  categories 
These  parameters  are  a  cost  improvement  curve  rate,  production 
rate,  and  first  unit  cost.  In  addition,  two  parameters,  opti¬ 
mal  curve  rate  and  start-up  costs,  were  necessary  to  represent 
those  alternatives  having  a  second  source. 

The  first  three  parameters  are  derived  from  program 
lot  quantities  and  costs  using  TASC's  Learning  Curve  and  Pro¬ 
duction  Rate  (LCPR)  Model.  LCPR  is  TASC's  surface  fitting 
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model  that  solves  for  cost  improvement  rate,  production  curve 
rate,  and  first  unit  cost  simultaneously.  Using  exponential 
formulations,  LCPR  reaches  a  solution  based  on  a  least-squares 
fit.  It  obtains  this  by  using  a  log-linear  solution  as  an 
initial  starting  point  and  proceeding  through  successive  itera¬ 
tions  based  on  a  generalization  of  Newton's  method  for  finding 
the  roots  of  a  non-linear  function.  The  cost  improvement  curve 
and  production  rate  parameters  were  then  combined  with  other 
previously  computed  points,  divided  into  groups  of  similar 
strategy  alternatives,  and  their  average  computed. 

Derivation  of  the  optimal  curve  was  based  on  prior 
TASC  analysis.  TASC  has  found  that  under  competition,  pro¬ 
duction  lot  costs  are  below  the  normal  sole  source  curve.  In 
general,  the  optimal  curve  represents  the  learning  curve  a 
producer  must  follow  to  reach  these  lower  lot  costs  given  a 
sole  source  starting  point.  The  linear  relationship  between 
the  optimal  curve  and  the  sole  source  curve  found  in  previous 
analysis  was  applied  to  the  sole  source  learning  curve  average 
to  obtain  the  optimal  curve  for  this  analysis. 

Start-up  costs  were  assumed  to  be  any  costs  of  tech¬ 
nology  transfer,  new  tooling,  and  learning  quantities.  While 
few  complete  data  points  were  available,  a  trend  emerged  when 
start-up  costs  were  compared  to  their  respective  first  unit 
costs.  This  relationship  was  used  to  estimate  start-up  costs 
in  missile  systems. 
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APPENDIX  E 

RISK  ASSESSMENT  QUESTIONNAIRE 

A  key  component  of  the  prototype  model  is  a  relative 
measure  of  how  acquisition  strategies  reduce  program  risk. 

This  was  accomplished  through  the  use  of  a  questionnaire  in 
which  an  individual  knowledgeable  in  weapons  acquisition  rated 
the  ability  of  Phase  1  and  2  strategies  to  reduce  given  levels 
of  risk  to  selected  levels  of  risk. 


E.l  QUESTIONNAIRE 

The  attached  questionnaire  was  completed  by  a  limited 
number  of  individuals  with  direct  knowledge  of  missile  and/or 
electronics  acquisition. 


E . 2  DATA  ANALYSIS 

Due  to  the  limited  response,  analysis  was  based  upon 
the  highest  and  lowest  ratings  given  for  each  question,  rather 
than  an  average  of  all  responses.  More  data  would  allow  a 
more  rigorous  approach.  The  average  of  these  two  points  was 
normalized  to  1.00.  See  Table  E-l  and  Table  E-2.  This  gives 
the  probability  of  success  of  reducing  risk  from  a  perceived 
pre-phase  status  to  a  desired-end-phase  status  for  the  general 
strategy  alternatives. 

This  information  was  mapped  onto  a  10  x  10  matrix  for 
each  strategy  alternative  and  each  level  or  risk  (see  Figure 
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E-l).  These  mappings  revealed  a  similar  trend  in  every  alter¬ 
native  and  category.  All  mappings  showed  approximately  the 
same  probability  of  reduction  from  risk  level  5  to  risk  level 
3  as  from  7  to  5 ;  and  the  same  from  7  to  3  as  9  to  5.  Given 
this  pattern,  it  was  assumed  to  hold  for  all  pre-phase  to 
end-phase  risk  levels.  The  risk  reduction  probabilities  from 
any  risk  level  1  through  10  to  any  lower-risk  level  were  esti¬ 
mated  from  these  points  (see  figure  E.2).  A  similar  mapping 
for  each  alternative  in  Phases  1  and  2  for  each  of  the  three 
levels  of  risk  was  developed  and  incorporated  into  the  proto¬ 
type  model. 
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Moderate  disruption  No  disruption  .25  .43 

Slight  disruption  No  disruption  .37  .68 
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Figure  E-l:  Software  Dependency/Prototype  -  Probability 
of  Risk  Reduction/Selected  Levels 


E-5 


sr  _ 


l 


THE  ANALYTIC  SCIENCES  CORPORATION 


Figure  E-2:  Dependency/Prototype  -  Probability  of 
Risk  Reduction/All  Levels 
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RISK  ASSESSMENT  QUESTIONNAIRE 
ATTACHMENT 
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INSTRUCTIONS 


INTRODUCTION 

Ve  Need  Your  Help 

The  Analytic  Sciences  Corporation  (TASC),  under  con¬ 
tract  to  the  Defense  Systems  Management  College  (DSMC),  is 
developing  an  analytic  model  for  use  in  acquisition  strategy 
decisions.  The  initial  objective  is  an  effective  teaching  aid 
for  evaluating  strategy  options  in  the  research,  development, 
and  production  phases  of  weapon  systems  acquisition.  Should 
this  prove  successful,  a  long-term  objective  is  an  acquisition 
strategy  decision  aid  for  program  managers. 

Included  in  the  analytic  model  is  the  concept  of 
technical  risk  through  various  phases  of  the  acquisition 
process.  One  method  of  quantifying  technical  risk  is  through  a 
subjective  assessment  by  a  broad  range  of  system  acquisition 
experts.  Several  methods  were  considered  before  a  straight¬ 
forward  rating  of  each  alternative  was  chosen. 

This  questionnaire  is  being  distributed  to  persons 
knowledgeable  in  the  acquisition  field.  After  receiving  their 
opinions,  the  results  will  be  integrated  to  determine  relation 
ships  among  acquisition  strategies,  as  they  pertain  to  risk. 
Your  assessment  of  the  situations  described  in  this  question¬ 
naire  is  very  important  and  will  greatly  enhance  the  validi  11 
of  the  research  effort.  Specific  questions  or  comments 
regarding  the  questionnaire  should  be  addressed  to  Mr.  Larry 
Cox  or  Ms.  Michal  Bohn  of  TASC  at  (703)  558-7400.  Comments  or 
questions  regarding  the  general  nature  of  this  research  effort 
should  be  addressed  to  Mr.  John  McKeown  of  DSMC  at  (703)  664- 
5783  (Autovon  354-5783). 
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OBJECTIVE 

The  objective  of  this  questionnaire  if  to  collect  the 
baalc  research  data  to  help  determine  how  selected  acquisition 
strategies  reduce  certain  aspects  of  technical  risk  associated 
with  weapon  system  and  subsystem  development. 

APPROACH 

In  this  questionnaire,  you  are  asked  to  consider 
strategy  alternatives  under  given  conditions  of  technical  risk 
and  rate  the  strategy^  ability  to  reduce  the  given  technical 
risk.  A  discussion  of  the  concepts  and  terminology  used  in 
this  questionnaire  follows. 

ACQUISITION  PHASES 

Our  research  work  follows  Department  of  Defense 
Directive  5000.1,  subject  Major  System  Acquisition,  dated 
March  19,  1980  which  defines  the  four  principal  phases  asso¬ 
ciated  with  the  acquisition  process  of  major  defense  systems 
as : 


Phase  0:  Concept  Exploration  --  includes 
solicitation  and  exploration  of 
alternative  system  concepts. 

Phase  1:  Demonstration  and  Validation 
(D&V)  --  also  referred  to  as 
Advanced  Development. 


Phase  2:  Full-Scale  Development  (FSD) 

also  referred  to  as  Engineering 
Development  -•  includes  limited 
production  for  operational  test 
and  evaluation. 


Phase  3:  Production  and  Deployment. 


One  of  the  major  objectives  of  Phase  1  and  Phase  2 
is  the  reduction  of  technical  risks  or  uncertainty  to  an 
acceptable  level .  Only  these  two  phases  are  addressed  in 
this  questionnaire. 
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STRATEGY  ALTERNATIVES 

In  each  of  the  two  phases,  alternative  acquisition 
strategies  are  available.  The  options  of  interest  in  this 
questionnaire  have  been  summarized  into  three  alternatives 
during  Phase  1,  D&V ,  and  three  alternatives  during  Phase  2, 
FSD,  as  follows: 


PHASE  1:  DEMONSTRATION  AND  VALIDATION  (D&V) 


Alt 

1: 

Contract  Definition  -  Short,  intense 
planning  and  evaluation  to  determine  cost, 
schedule,  performance  specification, 
management  technique,  etc.,  for  FSD  phase. 

Alt 

2: 

Subsystem  and  Component  Development  - 
Critical  elements  of  a  system  are 
developed  and  major  subsystems  de¬ 
signed  and  tested. 

Alt 

3: 

System  Prototype  -  Design,  build, 
and  test  proposed  end  item  (pre- 
production  prototype). 

PHASE  2:  FULL-SCALE  DEVELOPMENT  (FSD) 

Alt  1:  Incremental  Development  -  Design 

completeness  and  reliability  con¬ 
firmed  prior  to  production. 

Alt  2:  Partial  Concurrency  -  The  initiation 

of  significant  tasks  related  to  pro¬ 
duction  (without  a  complete  first  lot 
order)  prior  to  the  end  of  FSD. 

Alt  3:  Full  Concurrency  -  Production  begins 

prior  to  the  completion  of  FSD  with 
a  complete  first  lot  order. 


NOTE:  In  this  context,  concurrency  is  characterized  by  the 

initiation  of  product ion -re la ted  tasks  prior  to  the 
completion  of  FSD.  The  degree  of  concurrency  employed 
in  specific  cases  may  vary  from  a  small  contract  which 
holds  the  contractor's  technical  staff  together,  to  a 
complete  first  lot  order. 
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RATING  SCALE 

For  each  question,  which  corresponds  to  a  given  state 
of  technical  risk,  you  are  asked  to  rate  the  strategy  alterna¬ 
tives  on  a  numerical  scale  which  indicates  how  successful  you 
feel  the  strategy  alternative  would  be  at  reducing  risk  as 
described.  The  rating  scale  is  as  follows: 


Rating 


Definition 


1  Alternative  is  incapable  of  reducing 

risk  as  described. 


3  Alternative  has  a  slight  change  of 

reducing  risk  as  described. 

5  Alternative  has  a  50/50  chance  of 

reducing  risk  as  described. 

7  Alternative  has  an  excellent  chance 

of  reducing  risk  as  described. 

9  Alternative  will  certainly  reduce 

risk  as  described. 


2, 4 ,6, 8  Intermediate  values  to  accommodate 

compromise  between  the  above  values. 


TECHNICAL  RISK 

For  the  purpose  of  this  survey,  technical  risk  is 
divided  into  three  generic  categories: 


•  Level  of  Technology  Advance  (Hardware) 

The  existing  level  of  technology  in 
industry  is  constantly  subject  to  change. 
An  item  considered  to  require  new  and 
radically  different  system  design  today 
may  well  be  considered  contemporary  tech¬ 
nology  in  a  few  years.  The  concept  em¬ 
bodied  in  this  category  is  the  magnitude 
of  the  technology  increase  over  the  exist¬ 
ing  state-of-the-art. 


* 

l 


I 


4 


THE  ANALYTIC  SCIENCES  CORPORATION 


•  Degree  of  Required  System  Integration 

A  large  weapon  system  with  many  complex 
internal  and  external  interfaces  is  a 
high-technology  risk  program;  not  neces¬ 
sarily  because  it  embodies  advanced  techno¬ 
logy.  but  because  it  is  vulnerable  to  a 
large  number  of  error  sources. 

•  Level  of  Software  Dependency 

A  weapon  system  using  off-the-shelf 
components  with  few  intefaces  may  stll  be 
dependent  on  a  large  computer  software 
development  effort.  If  the  software  is 
critical  to  the  operation  of  the  system, 
its  development  could  pace  the  devel¬ 
opment  of  the  entire  system. 


Within  each  category,  five  degrees  of  technical  risk 
have  been  defined,  ranging  from  low-risk  (A)  to  a  high-risk 
(E)  situation.  The  definitions  of  these  degree  levels  are  as 
follows: 


Cat  1:  Level  of  Technology  Advance  (Hardware) 
Level 


A  -  None  (shelf  item) 

B  -  Minor  development  required  (at  least  one 
subsystem  needs  some  improvement) 

C  -  Moderate  development  required  (at  least  one 
subsystem  requires  major  improvements) 

D  -  Major  development  required  (several  sub¬ 
systems  require  major  improvements) 

E  -  Total  redesign  (all  new  technology  must 
be  developed) . 


-  5  - 
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Cat  2;  Degree  of  Required  System  Integration 
Level 

A  -  Hone  (item(s)  can  be  "plugged  in") 

B  -  Minor  redesign  (at  least  one  interface  needs 
some  redesign) 

C  -  Moderate  redesign  required  (at  least  one  inter- 
face  needs  major  redesign) 

D  -  Major  redesign  (several  interfaces  need  major 
redesign) 

E  -  Total  redesign  (all  interfaces  must  be  totally 
redesigned) 

Cat  3:  Level  of  Software  Dependency 
Level 


A  -  None  (delays  cause  no  disruption  of  hardware 
development) 

B  -  Minor  (delays  cause  slight  disruption 
of  hardware  development) 

C  -  Moderate  (delays  cause  moderate  disruption 
of  hardware  development) 

D  -  Major  (delays  cause  major  disruption  of 
hardware  development) 

E  -  Total  (magnitude  of  effort  so  large  that  delays 
cause  total  disruption  of  hardware  development) 


-  6  - 
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EXAMPLE i 

Phase :  D&V 

Risk  Category:  Level  of  Technology  Advance 

State:  Perceived  Pre-phase  Status  --  Total  Redesign 

Desired  End-Phase  Status  --  Moderate  Develop¬ 
ment  required 

If,  based  upon  your  experience  and  judgment,  you 
feel  that  contract  definition  is  incapable  of  achieving  the 
desired  result,  that  subsystem/component  development  has  a  very 
slight  chance,  and  that  a  system  prototype  has  an  excellent 
chance  of  achieving  the  desired  result,  the  question  would  be 
answered  as  follows: 


STATE 

ALTERNATIVES 

Perceived 
Pre-Phase  Status 

Desired 

End-Phase  Status 

Contract 

Definition 

Subsystem/ 

Component 

Development 

System 

Prototype 

Total  Redesign 

Moderate  Devel¬ 
opment  required 

1 

2 

7 

-  7  - 
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RISK  CATEGORY:  LEVEL  OF  TECHNOLOGY  ADVANCE 


RISK  CATEGORY:  DEGREE  OP  REQUIRED  SYSTEM  INTEGRATION 


1st  one  Interface  At  least  one  Interface 
■ajor  redesign  needs  nose  redesign 
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RISK  CATEGORY:  LEVEL  OF  SOFTWARE  DEPENDENCY 


Moderate  disruption  1  No  disruption 


